Add endpoint for purl lookup - #1359

@TG1999 I tried to test this new PURL lookup endpoint but cannot find the branch lookup_endpoint anywhere in the nexB repo. How do I access that branch? Or is that branch not in the nexB vulnerablecode repo (where I expect it should be)? It almost looks like the branch is in a fork on your own repo, but I don't know why that would be the case -- all of our work should be in the nexB vulnerablecode repo, right?

@TG1999 BTW, I've been doing some initial testing of our API bulk PURL search endpoint and other search endpoints/methods -- using pkg:nginx/nginx@0.8.9?os=windows which we know has two duplicate records with the same PURL -- and have found that:

• my local .py single-PURL API search returns 3 PURLs: 2 dupes of pkg:nginx/nginx@0.8.9?os=windows plus pkg:nginx/nginx@0.8.9

• my local .py bulk-PURL API search returns 2 PURLs: 2 dupes of pkg:nginx/nginx@0.8.9?os=windows -- just what we want (except for the duplication) i.e., only the PURL I searched for -- works for a single-PURL search so maybe we don't need a new lookup endpoint?

• public Package UI single-PURL search returns 3 PURLs: 2 dupes of pkg:nginx/nginx@0.8.9?os=windows plus pkg:nginx/nginx@0.8.9

• public API UI single-PURL search (GET /api/packages/) returns 3 PURLs: 2 dupes of pkg:nginx/nginx@0.8.9?os=windows plus pkg:nginx/nginx@0.8.9

• public API UI bulk-PURL search (POST /api/packages/bulk_search/): throws an error

Unable to get this to work.

The required input template differs from the structure that works in my local .py file.

{
  "purl": "string",
  "type": "string",
  "namespace": "string",
  "name": "string",
  "version": "string",
  "qualifiers": {
    "additionalProp1": "string",
    "additionalProp2": "string",
    "additionalProp3": "string"
  },
  "subpath": "string"
}

I tried this without success:

{
  "purl": "pkg:nginx/nginx@0.8.9?os=windows",
  "type": "nginx",
  "namespace": "",
  "name": "nginx",
  "version": "0.8.9",
  "qualifiers": {
    "os": "windows"
  },
  "subpath": ""
}

Response:

{
  "Error": "A non-empty 'purls' list of PURLs is required."
}

I also tried the following structure without success (but this DOES work in my local .py file AND is recommended in our RTD (https://vulnerablecode.readthedocs.io/en/latest/api.html#package-bulk-search)):

{
    "purls": ["pkg:nginx/nginx@0.8.9?os=windows"],
    "purl_only": False,
    "plain_purl": False,
}

Response:

{
  "detail": "JSON parse error - Expecting value: line 3 column 18 (char 70)"
}

Does the public API UI bulk-PURL search (POST /api/packages/bulk_search/) actually work and I just can't follow directions? ;-)

@johnmhoran see https://github.com/nexB/dejacode/blob/2bdb96d1fefca88e10db4c4e0c8efdcecb05864f/dejacode_toolkit/vulnerablecode.py#L69 how bulk_search API is used here

@TG1999 I don't think you read my comments very closely. I already have the bulk search API working in my Python code -- but I am unable to get the Public API UI bulk search working (here: https://public.vulnerablecode.io/api/docs/), and I explained what the experience was with that. I also do not know where your lookup branch is. All of that is detailed in my comments above, including my successful .py use of this structure:

{
    "purls": ["pkg:nginx/nginx@0.8.9?os=windows"],
    "purl_only": False,
    "plain_purl": False,
}

Added bulk_lookup endpoint with tests, please review

@TG1999 I've added your repo as a new remote and can now access your lookup_endpoint branch. So far I've focused only on the new lookup endpoint.

Based on the tests, I expected this structure to work with my local instance:

headers = {"Authorization": f"Token {token}"}

request_body = {"purl": "pkg:nginx/nginx@0.8.9?os=windows"}

response = requests.post(
    "http://127.0.0.1:8001/api/packages/lookup",
    data=json.dumps(request_body),
    content_type="application/json",
    headers=headers,
).json()

However, this threw an error:

TypeError: Session.request() got an unexpected keyword argument 'content_type'

Not sure why that works in your tests but not for me. Googling was unhelpful, and the variations I tried in the key 'content_type' did not resolve the error. I finally deleted that line from the requests.post() content and instead modified my headers like this:

headers = {"Authorization": f"Token {token}", "Content-Type": "application/json"}

and the following returned the expected list containing two duplicate pkg:nginx/nginx@0.8.9?os=windows PURLs:

request_body = {"purl": "pkg:nginx/nginx@0.8.9?os=windows"}

response = requests.post(
    "http://127.0.0.1:8001/api/packages/lookup",
    data=json.dumps(request_body),
    headers=headers,
).json()

I'll continue my exploration of the lookup endpoint and then turn to the bulk_lookup endpoint.

@TG1999 Very nice! 👍